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A  Geometric  Algorithm  for  Solving 
the  General  Linear  Programming  Problem 

§1.  Intr5duction 

The  development  of  the  subject  of  linear  programming  has 
centered  to  a  large  extent  on  the  problem  of  maximizing  (or  mi- 
nimizing) a  linear  form  the  variables  of  which  are  subject  to 
linear  inequalities  or  constraints.   There  have  been  proposed 
many  procedures  for  carrying  out  such  maximizations  in  an  ef- 
fective way,  foremost  of  which  is  the  simfllex  method  of  G.  Dant- 
zig.  All  of  these  methods  have  the  objectionable  feature  that 
on  occasion  the  number  of  steps  required,  though  finite,  be- 
comes excessively  large.  Because  of  this  (and  in  the  absence 
of  any  alternative)  the  attitude  assumed  by  many  workers  in  the 
field  is  that  a  large  collection  of  methods  should  be  evolved 
in  the  hope  that  on  all  occasions  at  least  one  of  them  will 
prove  practicable.   In  this  spirit,  a  method  is  proposed  in 
this  paper  which  is  essentially  a  "gradient"  procedure,  and  yet 
has  the  added  feature  of  concluding  in  a  finite  number  of  steps. 
In  some  problems  it  works  more  "efficiently"  than  the  simplex 
method  as  a  consequence  of  the  fact  that  instead  of  moving  be- 
tween neighboring  vertices  of  the  polyhedron  of  "feasible  solu- 
tions" (as  in  the  simplex  method)  it  provides  for  moving  across 
faces  of  this  polyhedron. 


82,  The  linear  programming  problem.    Let: 

E„  denote  euclidean  n-space: 

n  '         ' 

A   denote  an  mxn  matrix  viith  rows  R,  .....R    ; 

J.*    *  m* 

b  =  (b, , .. »,b  )  be  an  m-dimensional  (column)  vector; 
P  =  (Pt» * » • tPn)    De  an  n-dimensional  (row)  vector. 

Then  the  general  linear  programming  problem  may  be  stated 

as  follows: 

Find  x£E„  such  that  x  maximizes 
n 

L(x)  =  p.x  =  ?!*!+• ••+Pnxn  > 

subject  to  the  constraints 

Ax  >  b  . 
(Note:   The  non-negativeness  of  the  x.'s  need  not  be  among  the 
constraints .) 

§3*  Further  notation  and  its  geometrical  significance. 

Let: 

|x|  denote  the  length  of  a  vector  x  E  ; 

D  =  jx|x<c.  En  and  Ax  >  b  j  ; 

D*  =  i  x|x£  D  and  L(x)  >  L(u)  for  all  u€D-'  ]  . 
The  following  is  well  known  and  easily  verified. 

Theorem  1.  D  and  D"  are  closed  and  convex. 

If  xGD  ,  then  we  call  x  a  solution  to  our  problem.   If 
x€.D,  then  we  call  x  a  feasible  solution. 


Let  A  be  an  rxn  submatrix  of  A  whose  rows  are  R.  ,  ...,R.,  • 

Corresponding  to  this  submatrix,  let: 

<v 

b  be  the  (column)  vector  (b.  ,  ...,b.  ); 

xl      xr 


P  =  1  x I x  £  D  and  Ax  =  b  > ; 

rO     t  ^ 

V  =  J  x  I  x  e  E  and  Ax  =  0  V  ; 
p  be  the  projection  of  p  on  V; 
p.p/|p|  if  p  ?   o  • 

q  s  ~ 

0       if  p  =  0 


'Xs 


(Note:  We  allow  r  =  0,  in  which  case  P=D,  V=E,  p=p,  and 

q  =  |p|.) 

Before  launching  our  discussion  of  the  algorithm,  which 
will  be  presented  in  a  completely  algebraic  context,  it  will  be 
worthwhile  to  give  geometric  meaning  to  the  symbols  above,  and 
to  give  a  geometric  description  of  the  algorithm.  For  it  is 
the  geometry  that  motivates  the  method,  and  by  interpreting  our 
results  geometrically  at  each  stage  of  the  discussion,  the  rea- 
der should  have  no  trouble  following  along. 

As  it  is  well  known,  D  is  the  intersection  of  halfspaces, 
and  therefore  a  polyhedron  (which  may  be  bounded,  unbounded  or 
even  empty) .  Then  we  may  interpret  F  as  being  a  face  of  this 
polyhedron  (which  may  be  empty) .   Now  suppose  we  have  a  point 
x6D,  For  this  x,  L(x)  has  some  value.  Then  what  we  would 
like  to  do,  is  to  find  a  point  y6D  such  that  L(y)  >  L(x)  • 
Furthermore,  it  would  seem  plausible,  in  moving  from  x  to  y,  to 
do  so  in  a  manner  that  changes  L(x)  as  much  as  possible  for 
each  unit  of  distance  we  move.  Then  of  course,  this  direction 


.  •  •  . 


we  move  in  will  be  the  gradient  direction  p,  i.e.,  if  we  move 
from  x  to  a  point  y  =  x+Ap  where  X  >  0,  we  will  maximize  L(x) 
as  much  as  is  possible  for  each  unit  of  distance  moved.   Of 
course,  in  general,  we  cannot  move  as  far  as  we  like  in  this 
direction,  since  we  may  move  out  of  the  polyhedron  D,   In  fact 
we  may  not  even  be  able  to  move  at  all  in  the  gradient  direc- 
tion. This  will  occur  for  example  if  we  are  in  certain  faces 
of  D.   In  this  case,  if  x  belongs  to  a  face  F,  then  it  would  be 
plausible  to  move  in  the  gradient  direction  of  that  face.  But 
the  gradient  direction  of  the  face  F  is  the  direction  p.  Fur- 
thermore the  rate  of  change  of  L(x)  per  unit  distance  in  the 
direction  p  is  q. 

We  now  describe  our  algorithm  geometrically.  For  ease, 
let  us  assume  for  the  moment  that  D  is  non-empty,  and  that 
p  ^  0,  Let  us  further  assume  that  we  have  a  point  W  6D,  Now 
we  find  a  face  F  such  that  W  €  F  and  such  that  we  can  move 
in  the  gradient  direction  p  ^  0  from  the  point  W  and  remain 
in  D.  We  now  assert: 

3.1)  If  W°  is  not  already  a  solution,  then  such  a  face 
exists.   If  no  such  face  exists,  then  W°£  D'"*'. 

Now  if  W  is  not  a  solution,  we  find  a  maximum  \   such  that 
W  =  W  +Ap  £D.   I.e.,  we  move  as  far  as  we  can  in  the  direc- 
tion  p  .   Our  next  assertion  is; 

3.2)  If  a  maximum  A  does  not  exist, then  the  problem  has 
no  solution. 

This  is  easily  seen,  for  in  this  case,  W°+Ap°6D  for  all 


X  >   0,  and  L(W°+Ap°)  — >oo  as  A  — >oo  (since  p°*p  >  0). 

Now  if  either  3*1)  or  3.2)  occurs,  we  have  solved  the 

problem.   If  not,  we  start  with  the  point  W  and  go  through  the 

""1  1  ~1 

same  process.   I.e.,  we  find  a  face  F  such  that  VI  f  F  and 

such  that  we  can  move  a  positive  distance  in  the  direction 
p   (/  0),   If  there  is  no  such  face,  then  again  we  are  at  a 

solution.   Otherwise  we  find  a  maximum  \x   (if  it  exists)  such 

2     1   vl 

that  Wfc  =  W  +up  £  D.   If  3.2)  applies  in  this  case  then  as  be- 
fore no  solution  exists  and  we  need  not  go  further.  Otherwise 

2 

we  continue  the  process  with  W  .  Finally  we  assert: 

3,3)  In  a  finite  number  of  steps  the  process  will  termi- 
nate. I.e.,  for  some  integer  t,  we  will  reach  a  point  W  such 
that  either 

a)   we  cannot,  for  any  F  containing  W  ,  proceed  in  the  di- 

**>  t 

rection  p,  in  which  case  W  is  a  solution;  or  b)   at  the  point 

W  ,  3»2)  will  apply,  in  which  case  there  is  no  solution. 

The  rest  of  this  paper  will  attempt  to  make  the  above 
ideas  sharper,  and  to  prove  the  three  assertions. 

Before  we  close  this  section,  we  will  make  one  more  point. 
Of  the  three  assertions,  the  first  one  is  the  most  subtle  one. 
Namely,  suppose  we  are  at  a  point  VI 6  D,  such  that  VI  is  not  a 
solution.  Then  by  the  convexity  of  D,  we  know  there  is  some 
direction  we  can  move  in  (and  still  remain  in  D)  that  will  im- 
prove the  value  of  L(x) .  Assertion  3.1  says,  that  p,  the  gra- 
dient  direction  of  a  face  F,  is  among  these  directions.   Physi- 
cally this  is  clear.  For  suppose  p  is  the  downward  direction. 


Then  we  can  place  a  pebble  at  the  point  W  and  have  it  fall 
freely  in  the  direction  p.   If  we  now  contrain  the  pebble  to 
the  polyhedron  D,  then  it  is  clear  that  it  will  start  falling 
in  the  gradient  direction  of  some  face  (which  may  possibly  be 
D  itself).   Thus  on  physical  grounds,  assertion  3«1)  should 
come  as  no  suprise.  However,  it  does  remain  to  be  proved  ma- 
thematically, and  our  next  section  devotes  itself  to  such  a 
proof. 

§ij..  The  gradient  directions  as  directions  of  motion. 

Let  x,x£E   .     Then  we   define 

'  n 

(  IdEhMzl    lf    5c  *  x  . 
A(x,x)    -  <       f5-x> 

0  if     x  =  x 

We  may  note   that   if  x  ^  x,    then 

A(x,x)   =  p»f§=Sl     . 
|x-x| 

We  interpret  /\^(xtx)    to  be  the  change  of  L(x)  per  unit  distance 

as  we  move  from  x  to  x.  We  also  observe  that  /^(x  ,x)    depends 

only  on  the  direction  x-x  and  not  on  the  distance  from  x  to  x. 

I.e.,  if  x  =  x+?ty  and  x»  =  x+?v'y  inhere  X,A»  >  0  and  y  /  0,  then 

/\(x,x)  =  /\^(x',x).  Furthermore  if  x  =  x+Ap  where  \  >   0  and  p 

is  the  gradient  direction  of  the  face  P,  then  /\(x,x)  =  q. 

Since  p  is  the  maximum  rate  of  change  of  L(x)  per  unit  distance, 

the  following  lemma  should  be  quite  plausible. 

Lemma  1.   If  x,x  £F,  then  /^(x ,x)  <  q. 


Proof,  We  may  write  p  uniquely  as  p  =  p+s  where  p£V  and 
sGV-J*.  Thus  for  any  vector  y£V,  y»p  =  y»p.  Now  if  x  =  x, 
then  £\(x,x)  =  0  <  q.   If  x  /  x,   Then  x-x/|x-x  |€  V,  hence 


^(x,x)  =  p.IE=si  =  j;lfcal  <  3.  JL  =  p.jL  =  $  . 

|x-x|      |x-x|      |p|      |pf 
Definition.  Por  x 5  D,  let 

Sx  =julu^En'  'ul  =  l5  and  for  some  ^  >  °»  x+Au6Di. 
(Note:   In  the  above  definition  A  depends  on  u  as  well  as  x.) 
We  may  interpret  S  as  the  set  of  directions  we  may  move 

■A 

from  the  point  x  and  still  remain  in  D,  Then  what  we  are  aim- 
ing for,  is  to  show  that  if  x€.D-D",  then  there  exists  a  gra- 
dient direction  p  for  some  face  such  that  p/|p|££  . 

Lemma  2,  Por  x  D,  £   is  compact. 

Proof.  Let  I  =  l,...,mi'.  Let  J  =  J  i|i£  I,  and  R^x  =  b^j  • 
Then  for  i(LI-J,  R..x  >  b..  Now  for  i€  I-J,  there  exists 
%s    >   0,  such  that  given  any  unit  vector  u€E  ,  and  any  T)   such 
that  0  <tj<  \,   Ri«(x+Au)  >  b±.  Let  A  =  iTJ™^  \±,     Now  for 
i£«J,  and  u€£x,  R.«(x+Au)  >  b..  Thus  for  this  A,  x+Au  CD  for 
all  u^Sx.   Thus  if  u^€.  S   (i=  1,2,3,...)  has  the  limit  point 
u,  then  x+Aun  — >  x+Au.  But  x+Au  nS  D  and  D  is  closed,  hence 
x+Au£D.  Therefore  u£  £  ,  showing  £  is  closed.  But  S  is 
also  bounded,  therefore  compact. 

Lemma  3.  Por  x£D-D*"',  there  exists  u€S   such  that 

u«p  >  0.   (Note  we  are  assuming  D-D"  to  be  non-empty  which  im- 

— > 
plies  p  ^  0.) 
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Proof.  Since  x£d-D",  there  exists  y€  D  such  that  L(y)  - 
-  L(x)  >  0.  Let  u  =  (y-x)/|y-x|.  Then  it  is  clear  that  u  has 
the  desired  properties. 

Lemma  u..  For  xCD-D",  there  exists  a  unique  vector  u  Q.  S 
such  that 

X 

Proof.     By  lemma  2,   at  least  one   such  u"  exists.     Now  sup- 
pose  there  were   two    such  vectors   a?  and   Up.     Then  by  Lemma  3, 

Ul  ^  "u2*      Thus   °  <    lul+upl    <  2#     Now  for  some  ^1»    ^2   >  °* 
x+iu"J£D  and  x+AgUgED.     Let   \  =  min(A1,A2).     Then  x+Auj£D 

(i  =  1,2).      Then  by   the  convexity  of  D, 

|[x+Au*]+  |[x+Xu|j   =  x+  |(u£+uJ)£D   . 

Thus    ( u*+u* ) / | U*+u| [ €  Sx •     But , 

#,  «     .   *.  -x- 
n   Ul   2   >  n.  Ul   2  -  «  »* 

p.  ~ -   >  p»  ^ p.U,   , 

|u1+u2|        2 

which  contradicts  the  definition  of  uT. 

Definition.  For  xfcr.D-D  ,  we  denote  by  u  (x)  the  unit  vec- 
tor whose  existence  and  uniqueness  was  covered  by  Lemma  ij.. 

Lemma  5«  Suppose  D  is  non-empty.  Then  x6Dw  if  and  only 
if  S„  is  empty  or 

™H      (P.u)  <  0  . 
x 

Proof.  Follows  easily  from  Lemmas  2  and  3. 

Theorem  2.  Let  x<ED-DY  Then  there  exists  a  face  F  (whose 

gradient  direction  is  p) ,  such  that  xG  F  and  u*""(x)  =  p/|p|. 


.    v    \. 


Proof,  As  in  Lemma  2,  let  I  =j  l,...,mS,  and  let 

J  =  4i|R.»x  =  b.V.  Then  there  exists  X  >  0,  such  that  for  any 

i£.I-J,  and  any  unit  vector  u,  R,«(x+mi)  >  b.  where  0  <  r\  <  X, 

Now  for  some  A',  x+A'u*(x)  £D.  Let  £  =  minCX,?^')  and 

y  =  x+£u*'(x).  Then  y^D,  and  for  i  €  I-J,  R^y  >  b^.  Let 

J'  =^i|Ri.y  =  bA.     Then  J' C  J.  Let  J>  =  ji1,i2»  •  ••  >lr\>   let 

A  be  the  rxn  submatrix  of  A  whose  rows  are  R4  . ••..R-i  ,  let  F 

be  the  face  corresponding  to  A,  and  let  p  be  the  gradient  direc- 

tion  of  P. 

— > 
We  first  show  that  p  ^  0  .  By  Lemma  3,  L(y)-L(x)  = 

=  (p»u"(x))  >  0.  Thus  by  Lemma  1, 

o  <L(y)-L(x)  =  A(y>x)  <-. 

Thus  by  definition  of  q,  p  ^  0. 

We  now  show  that  for  sufficiently  small  e  >  0,  y+ep£D. 
Clearly  we  may  choose  e  so  small  that  Ri»(y+ep)  >  b.  for  i^I-J't 
But  for  all  positive  e,  and  all  iGji,  R^y+sp)  =  bi.  Thus 
such  an  e  exists. 

Now  let  w  =  £u'""(x)+ep.  Clearly  w  ^  0,  since  p»w  =  £(p«u"(x)) 

+e(p«p)  >  0.  Thus  w/|w|  ie  a  unit  vector,  and  x+|w| (-^— )  £  D. 

|w| 
Thus  —  €S  . 

Finally,  we  show  that  assumming  u"(x)  ^  -?r-»  we  get  a  con- 

IpI 

tradiction.  For  in  this  case, 

R^E>P'?*(*)  =  p.u*(x) 
|p|    K(x)f 

since  p  is  the  gradient  direction  for  the  face  F  (whether  we 


.   , 
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can  move  in  this  direction  or  not) .   Thus 

P'7T  =  7^-(P*u'::'(x))+  -Mp.p)  >  -^(p'u'^x))*  lLEl(p.u*(x)) 
M    |w|  |w|        |w|  |w| 

=  ■,  l?|g|   r(P«tt*(x))  >   ;  g+e»PJ (p.u*(x))  =  p.u*(xK 

|£u'(x)+ep|  5|u  (x)  |+e|p| 

Thus  p»(w/|w[)  >  p«u  (x)  which  contradicts  our  definition  of 
u  (x)  (since  (w/|w[)£S  ).  Thus  we  must  conclude  that  u*(x)  = 

=  p/IpI. 

Corollary  1.  For  x€D-Dw,  there  exists  y€D  such  that 
y-x  =  Ap  for  some  p  and  some  X   >  0.  Furthermore  for  this  y, 


§5.  The  geometric  algorithm. 

We  now  give  the  algebraic  formulation  of  the  algorithm 
which  we  described  geometrically  in  Section  3.  So  suppose  we 
are  given  the  constraints 

Ax  >  b 
and  we  wish  to  maximize 

L(x)  =  p*x  . 
We  further  make  the  assumption  that  we  are  given 

w°£D  . 
Of  course  this  is  a  strong  assumption,  but  in  Section  7,  we 
will  illustrate  (using  this  geometric  algorithm)  how  one  ob- 
tains such  a  w°  (if  it  exists).  We  of  course  also  know  the  di- 
rections  p  corresponding  to  the  faces  F.   (These  directions, 


11 


depend  only  on  the  matrix  A  and  the  vector  p.  A  discussion  of 
how  they  are  computed  will  be  left  for  Section  8.) 

Starting  with  w°,  we  now  define  a  sequence  of  points,  ei- 
ther finite  or  infinite  as  follows:   (We  use  induction) 

Suppose  w  is  already  defined  (k  >  0) .   Let  P,,,..,Pfl  be 
all  the  faces  containing  w  .  Let  p,,...,pn  be  the  gradient 
directions  for  these  faces.  Then  either 

/V   .  i  /-.J 


I)      Pj/IpjI  €fc  S  k  for   j  =  !,..,,£     (Pj^O). 


or 


II)  There  exists  j  (1  <  J  <  f )  sucn  tnat  P-  7*  °  and 

p./|pjesk.  j 

j     j  r       ~       i 

If  II)  holds,  let  L  =  ■.  i|Ri«p-  <  0V  .  Then: 

J  k 

a)   If  I)  holds,  terminate  the  sequence  at  w  . 


b)   If  II)  holds,  and  L  Is  empty,  terminate  the  sequence 


at  w  . 


c)   If  II)  holds,  and  L  is  non-empty,  define 

wk+1  =  wk+Xp. 
3 
where  k 

,    min   VVW 

1£L   Vp- 

Theorem  3.   If  the  sequence  terminates  at  w  ,  because  of 
a),  then  w  is  a  solution. 

Proof.  Follows  from  Theorem  2  and  its  corollary. 

Theorem  k*      If  the  sequence  terminates  at  w  because  of  b), 
then  there  is  no  solution. 


. 


I  . 
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Proof;   Since  II)  holds,  p-  ^  0.  Thus  p.p-  >  0,  hence 
L(w  +up-)  — >  oo  as  [i   — >  co  .  But  under  b) ,  FL'P-  >  0,  hence 

1    J  1        J 

^•(vT+np-)  >  b,  for  all  i  and  u.  >  0.  Thus  w  +u.p-)£  D  for  all 
J  J 

\i  >   0,  hence  the  problem  has  no  solution. 

In  view  of  our  definition  of  the  sequence  w  ,w  ,w  ,..., 
and  in  view  of  the  last  two  theorems,  it  follows,  that  if_  i*e 
can  give  a  procedure  for  constructing  the  sequence,  and  if_  we 
can  prove  that  the  sequence  always  terminates  in  a  finite  num- 
ber of  steps,  then  we  will  have  an  algorithm  for  solving  the 
programming  problem.  We  will  accomplish  this  shortly,  but 
first  we  shall  prove  a  result  which  will  shed  some  light  on  how 
this  procedure  is  comparable  with  our  geometrical  description. 

It  is  clear  that  if  w  and  w    are  two  consecutive  points  in 

k+1    k  *w' 
our  sequence,  where  w    =  w  +Ap»,  then  we  did  move  in  the  gra- 

k+1       k 
dient  direction  of  some  face  to  obtain  w    from  w  .  We  would 

like  now  to  show,  that  we  actually  have  moved  as  far  as  possible 

in  the  direction  p«. 

k      k+1    k  '"-, 
Theorem  5»  Let  w  and  w    =  w  +Ap-  be  two  consecutive 

k  — '  • 
points  in  the  sequence.  Then  X   >  0,  and  for  u-  >  A,  w  +|jp-<|£D. 

J 

Proof.  Since  'p-/|p-|6S  ,   there  exists  T]  >  0,  such  that 

z —  j     j     w 

w  +Tjp-GD.   Now  consider  some  i£L.  Then  R^'P-  <  °»  But  for 

this  i,  R1»(wk+Vjp-)  >  b±.  Thus  T]  (R^p-)  >  b^R^w  .  Hence 

k  J 

y         <  . -_- , 

I     R1.p- 

J 

But  this  holds  for  all  i£L.   Thus 


r: 
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0  *  *?£  iei)    ■    ^    >  -x  . 


I    hi?5j 


Hence  A  >  0, 

Now  suppose  n  >  A,  Let  I€L  be  such  that 


X  = 

Rj.p. 

Since  RT»P-   <  0. 
1      J 

M-(Rj«P?)  <  MRj'Pv)  =  bj-Rj.w 
Thus 

H(Rj.p,)+Rj.wk  <  bj 


k 


thus 


Rj^w^+np.,)  <  bj  . 


k  ^  »tH 

Hence  w  +up?  does  not  satisfy  the  I   constraint. 

Corollary  2.   |wk+1-wk|  >  0. 

Corollary  3.   If  w   ,w  ,w    are  three  consecutive  points 

of  the  sequence,  then  they  are  non-oollinear.   I.e., 

Iw^-w^fl^-w1*-1!  >  fw^-w^1! 


16.  The  finiteness  of  the  algorithm. 


Let  us  enumerate  all  the  faces  P  of  the  polyhedron  D,  for 
which  p  /0  (and  hence  q  >  0)  .  Namely  let  these  faces  be 

V1$Vzi*lij(h  (h  <  2m) 


where  0  <  q,  <  q2  <  . . .  <  q^. 


..•    ■■ 


■    ■ 


I 


" 


r .  »  - 


i      ; 


• 


Ik 


Let  the  corresponding  gradient  directions  be 

P^jPp' • * • »Ph  * 
Now  suppose  w°£D.  Let 

w^w^w2,...   (finite  or  infinite) 
be  the  sequence  defined  inductively  in  Section  5.  Let 

wk  =  w^^pj    (k  =  1,2,...) 

where  1  <  jk  <  h.  Then 

Lemma  6,  If  jk  =  j  h  (k  <  jf),  then  there  exists  s  such 

that  k  <  s  <  jl,   and  q  .  <  cf . 

J*?    ^k  „  , 

Eroof.  Let  P  =  P,  ,  p  =  pj  ,  q  =  q.  ,  Now  w  6  F  and 

W^e?.  Let  L  =  |w^wk|,  Lx  =  |wk+1.J|,  L2  =  fr*2-**"1!*** 
L„  k(-1  =  |w^  -w^~2|.  Then  by  Corollary  3, 

L  <  L, +Lp+. .  ,+Ln  ,  ..  • 

By  Lemma  1,  ^(w<"  ,w  )  <  q.  But 

Jk+1  i  Jk+2  d      J^-l  <"Kml 
Now  suppose  the  lemma  is  false,  i.e.,  q.      >   q  for  k  <  s  <  ^. 
Then 

qL  >  ^(w^"1,wk)L  >  'q(L1+.,.+Ln   fc-1)  ,>  qL  . 

Thus  we  have  a  contradiction. 

Now  let  p-  ,p .  >•••,  be  the  sequence  of  directions  (defi- 
«1   J2 

ned  as  before)  used  to  obtain  the  sequence  w  ,w  ,...  •  Let 
N(i)  be  the  number  of  times  p.  occurs  in  the  above  sequence 
(N(i)  could  be  co  ) .  Then 


■. 


.... 


r- 


*   .  i 


■ 


«. 


■  ■ 
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Theorem  6.  N(i)  <  2i_1  . 


Proof .  Suppose  that  ^ylp^y)  =  K  is  finite.  Suppose  fur- 

ther  that  p.  oocured  K+2  times.  We  will  then  get  a  contradic- 
ts/ 
tion.  For  suppose  p,  occured  in  the  sequence  in  the 

f    t    j         J 

1*2*  *3» • •  • »  K+2 

places.  Then  some  p  with  s  <  i  would  have  to  occur  between 

s 

the  a.  and  the  ^t+1  terms  of  the  sequence  by  Lemma  6*  Thus 

j~ vIiN(v)  >  K+l  which  is  a  contradiction.  Hence 

i-1 
N(i)  <  2Z  N(V)+1  . 

But  N(l)  <  1  by  Lemma  7.  Thus  a  simple  induction  argument  gives 
the  result. 


Corollary.  There  are  at  most  2  terms  in  the  sequence 


w 


o  ..1 


n 


,w  i . • .  .   (Thus  there  are  at  most  2^  terms  in  the  sequence.) 


§7.   Obtaining  a  feasible  solution. 

Let  us  assume  that  given  w  £-D,  we  know  how  to  compute 

1   2 

w  ,w  , . , .  and  we  are  able  to  determine  when  the  sequence  termi« 

nates.  Then  the  whole  algorithm  would  depend  on  obtaining 
w°€  D.  But  to  find  such  a  vector,  we  essentially  alter  the 
problem  as  follows.   Let  B  be  an  m-dimensional  column  vector 
all  of  whose  components  are  1,  Let  x  ,  be  a  new  variable. 
Then  consider  the  problem  of  maximizing 

L(x1,x2,...,xn+1)  ■  -xn+i  9 


^ 


.  '  ■■ 


. 


.» 


o 
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subject  to  the  constraints 

Ax+Bxn+1  ^  b  •  xn+l  ^  °  ' 

It  is  then  clear  that  a  feasible  solution  to  the  altered  problem 
exists,  namely,  x±  =  0  (i  =»  l,...,n),  x  -  =  max(0,blt., .,bm) . 
It  also  is  clear  that  a  solution  to  the  new  problem  exists, 
since  L, (x. , . ,. ,x  . )  is  bounded  above  by  zero.  We  may  then 
apply  our  algorithm  to  the  new  problem  and  obtain  a  solution 
x^, ...  »xn,x*n+1»  Then  x  =  (x,,...,x  )  will  be  feasible  for  our 
original  problem,  if  and  only  if  x +-  =  0. 

§8.  The  computations. 

To  carry  out  the  computational  scheme,  we  must  be  able  to 

1)  compute  p  for  a  face  P; 
and 

2)  decide  for  a  given  p  ^  0,  whether  or  ont  p/|p|€S  for 

W 

some  w€  D. 
That  these  can  actually  be  carried  out  is  shown  in  the  following 
two  results. 

Theorem  7.  p  =  p-AJ-u.  where  \i   is  any  solution  to  the  r 
equations 

Ap  =  AA-J-u.  » 
Furthermore  this  set  of  equations  always  has  a  solution. 

Proof;   We  may  write  p  =  p+s  where  p£.V  and  SEV-*-.  Pur- 
thermore  this  decomposition  is  unique.  Also  V  is  the  set  of 
vectors  in  E  ,  orthogonal  to  all  the  rows  of  A.   Thus  V*  is 


•: 


. 


■• 


..'-,  ,, 


.       '■■■; 


VS        >'. 


•  '! 


■■■; 


. 
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■ 
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generated  by  the  vectors  R±   , ....R*  .   Thus  we  may  write 

S  =  M^i-i*' *  *+^rRi  -=  A-Lp,  where  \i   =  (^t,,...,u.  ).   Then 

Ap  =  A(p+s)  =  As  =  AA-L^l.  Thus  the  equations  have  at  least  one 

solution. 

Now  let  ii  =  (h|»M||L)  be  any  solution  to  the  equations 
Ap  =  AA-J-n,  Then  let  s  =  AJ-ji  =  [i^R^  ♦•••+lJtrRi  •  Tnen  s€V-L. 
Now  A(p-s)  =  Ap-AA-J-ii  =0.  Thus  p-s€V.  But  p  =  (p-s)+s,  where 

p-s€.V  and  stLVJ-,  has  a  unique  such  decomposition.  Thus 

~     ^    7  f 

p  =  p-s  =  p-AJ-ji, 

Theorem  8»  Let  wfD.  Let  p  be  the  gradient  direction  for 

**  ~   j   "*"  (  i,   ~    ) 

P  (where  p/0),  Let  L  *   ji|R.«p  <  0  V  be  non-empty.  Let 


'V 


Then  p/|p|&Sw  if  and  only  if  X  >  0. 

Proof.  Suppose  X  >  0.  Then  w  =  w+Xp  is  distinct  from  Wi 
Now  for  i  ^E  L  R,.(w+Xp)  >  R.»w  >  b,.  Further  for  i£L, 
R4*(Xp)  >  b.-R.»w.  Thus  R,»(w+Xp)  >  b^.   Thus  w£~D.  Hence 
w+(X|p|)p/|p|£  D  where  X|p|  >  0.  Thus  p/|p|  GSW« 

The  implication  in  the  other  direction  follows  from  Theo- 
rem 6. 

Theorems  7  and  8  then  give  the  computational  procedure  for 
finding  the  sequence  w°,w  , ...  »   In  view  of  the  fact  that  we 
now  know  how  to  compute  the  sequence,  and  in  view  of  the  fact 
that  it  terminates,  we  now  are  able  to  determine  a  solution  or 
to  decide  that  a  solution  does  not  exist.  We  now  illustrate 
the  algorithm  with  an  example. 


-  t  *  •  * 


■4.  ^ 


\   '.  j<  i 
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§9*  An  example , 

Maximize 

L(x,y,z)  =  -2x-y+2z  , 
subject  to  the  constraints, 

2x  -z    >   -6  (1) 

-x      +lly     -5z   >   -30  (2) 

-13x    -22y     +z    >   -60  (3) 

y         >  o         (k) 

In  this  case  p  *  (-2,-1,2)  and  it  can  be  verified  that 
w°  -   (3,1,2)  is  a  feasible  solution  for  which  a  strict  inequa- 
lity holds  in  each  constraint. 

Now  we  let 

„1  _  ,,0.,  % 
w  -   w  +AiPn  • 

But  in  this  case,  p,  -  p  since  no  equality  holds.  Now 

P^'Hj  =  -6  ,  p^»R2  =  -19  ,  Pi»R3  =  50  ,  p^.R^  ■  -1  . 

Thus 


L,  =  ^ilpn.R4  <  0V=  jl,2,l4-! 
then 


_  min  ]   VRi-w°  )  _  Ww°  _  . 

Hence  w  =  w°+p1  «  (1,0,14.). 

Now  w  satisfies  the  equality  in  the  fourth  constraint. 
Thus  we  let 

A  a  (0,1,0)  . 


I  li 


V 


.    I 


f  •    • 

' I  -J  ■}     *      *  ) •      -! 
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Now  p2  =  P-|iRi.  where  \i   satisfies 

Ap  =  AAJ-n  . 
Thus  -1  =  ix,  hence  p2  =  (-2,0,2).  Then  w     -  w^+ApPp.  In  order 
to  compute  \~t   we  first  compute 

VR1  =  ~6  '   P2'R2  =  "8  '  P2#R3  =  28  •  VR1*  =  °  ' 
Then  Lg  *  >  llRj/P^  <  0  l«  J  1,2*1  ,  and 

.   f  b.-R^w1)   b.-R.^w1 

2       2i   V*2   J    V*2 
Thus  w2  -  (-1/3,0,16/3). 

Now  w2  satisfies  constraint  (1).  Therefore  we  try 

/■v 

A  =  (2,0,-1)  . 
We  then  set 

P3  =  P-^x 

where  ji  satisfies 

Ap  =  AA-Ljt  . 
Thus  |jl  =  -6/5.  Hence 


''V      P .     n. 
P3  =  (jj-ltf)  • 


Thus  w^  =  w2+A^p3  .  Now 


zv/  /v         77    ^>        88    ^ 

P3.RX  =  0  ,  p3.R2  b  --^  ,  P3-R3  =  j-  ,  P3«R^  =  -1   . 
Thus  L3  =  j  lfR±»p3  <  o|  =  \  2,l^|   ,   and 


min    i 


P  2 

b.-R.  «w  \       b,.-R.  »w 


^  -  mj-"  J  ±  ■*•   (  =  -at — it s  0  . 

3  ^L3/  VP3  J    V*3 
Thus  we  cannot  move  in  the  direction  p-  chosen.  Hence  we  try 


r 


aw  . . 


• 


•    •••    . 


■ 


i  ■  •   .-      i 


r   t. 


' 


*•• 


20 


to  move  in  a  different  direction.  Knowing  that  w  satisfies 
both  (1)  and  (lj.) ,  we  now  try 

km    I2    °  -1) 

A   ^0  1  o  '  • 


We  then  set 


where 


or 


?3  =  P'-liiR1-^2R2 
Ap  =  AAl(|i|) 

Thus  |a,  =  -  p-  and  n2  =  "1*  Then 

P3  =  (|,0f|)  . 

For  this  p~, 

R1.p3  =  0   ,   R2.p3  =  -^  ,   R3*p3  =  -^   ,   R,+  *p3  =  0 


In  this  case, 


L3  =  l2'3\ 


and  ,  9 

b„-R,«w* 


x,  -    2    £  ■     -  5/6  . 
^  R2,p3. 

Thus  w3  =  w2+X-p3  =   (0,0,6). 

Now  w^  satisfies  the  equality  in  (1),  (2),  (k) •  Thus 

there  are  eight  choices  of  A.  Now  for  most  of  these  A  we  can 

find  by  trial,  that  we  cannot  move  in  the  corresponding  direc- 

tion  p.  But  if  we  take 

A  =  (2   ?  "J) 


• 


■t 


. 


£    :-■. 


■  .- 
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21 
Then  we  find  that  p^  =  (5,5,3) .  Also 

Thus  L^  =  j  3  1  and  A.  =  12.  Hence  w^  =:  w3+A.  JT  a  (2,2,10)  . 

It  is  then  found  that  vr"  satisfies  the  equalities  in  (1), 
(2),  (3).  Thus  there  are  again  eight  choices  of  A.  Correspond 
ding  to  these  A,  we  have  eight  direction  p.   If  we  now  try  to 
move  in  any  of  these  eight  directions,  we  find  it  impossible. 
Thus  by  Theorem  2,  vF"   must  be  the  solution. 

§10.  Motion  in  the  "best  possible"  direction. 

In  Section  5,  we  showed  that  given  w°£D,  we  may  define  a 
sequence  (which  we  proved  to  be  finite) , 

where 


w  ,w  ,  ...,w 


wk  =  w^+Aj^Tj  (k=l,...,t). 

However  the  sequence  is  not  uniquely  defined,  since  at  each 

stage  we  may  have  a  choice  in  the  direction  p.  •  Now  suppose 

Jk 
we  specify  that 

P<  /IP-  I  =  u*(wK"1)  . 

Jk   Jk 

That  this  is  possible  is  clear  by  Theorem  2.   If  we  make  this 
choice  at  each  stage,  then  the  above  sequence  Is  uniquely  defi- 
ned. Furthermore  we  have  the  following: 

Theorem  9.   If  p*.  /Ifi  |  =  u'^w^1)  for  k  ■  l,...,t,  then 
~~      Jk   Jk 


•tti'J    '    . ; '■* 


• 


I 
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Proof,  Suppose  this  were  not  the  case.   Then  there  exists 
K  such  that  1  <  V  <   t-1  and  q,  <  q    .  Let  L,  =  \u*-u*~1\, 

Then  by  definition  of  u'^w*"1), 


£ 


/v^ 


B  P»U'*(w"~  )   . 

But  this  is  clearly  a  contradiction,  hence  we  conclude  the 
truth  of  the  theorem. 

Corollary  lu  If  p.  /|p,  |  =  u*^"1) ,  then  N(i)  <  1. 

Jk   Jk 

Thus  if  at  each  stage  in  the  sequence,  we  move  in  the  di- 
rection uw(w   ),  which  is  the  "best  possible"  direction  of  mo- 
tion (in  the  sense  that  we  increase  L(x)  as  much  as  possible 
per  unit  distance),  then  Corollary  I4.  says  that  we  move  through 
each  face  at  most  once.  Of  course  this  is  much  better  than  the 
result  of  Theorem  6.  However  in  computing  u"(w   )  at  each 
stage,  is  in  general  tedious.  The  above  situation  corresponds 
to  a  falling  pebble  constrained  to  a  polyhedra,  and  Corollary  \\. 
states  that  the  pebble  will  pass  through  each  face  at  most  once, 
which  substantiates  our  physical  intuition. 
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